<html>
  <head>
    <title>Debugging HDF5 Applications</title>

    <h2>Introduction</h2>

    <p>The HDF5 library contains a number of debugging features to
      make programmers' lives easier including the ability to print
      detailed error messages, check invariant conditions, display
      timings and other statistics, and trace API function calls and
      return values.

    </p><dl>
      <dt><b>Error Messages</b>
      </dt><dd>Error messages are normally displayed automatically on the
	standard error stream and include a stack trace of the library
	including file names, line numbers, and function names. The
	application has complete control over how error messages are
	displayed and can disable the display on a permanent or
	temporary basis. Refer to the documentation for the H5E error
	handling package.

	<br><br>
      </dd><dt><b>Invariant Conditions</b>
      </dt><dd>Unless <code>NDEBUG</code> is defined during compiling, the
	library will include code to verify that invariant conditions
	have the expected values.  When a problem is detected the
	library will display the file and line number within the
	library and the invariant condition that failed.  A core dump
	may be generated for post mortem debugging. The code to
	perform these checks can be included on a per-package bases.

	<br><br>
      </dd><dt><b>Timings and Statistics</b>
      </dt><dd>The library can be configured to accumulate certain
	statistics about things like cache performance, datatype
	conversion, data space conversion, and data filters. The code
	is included on a per-package basis and enabled at runtime by
	an environment variable.

	<br><br>
      </dd><dt><b>API Tracing</b>
      </dt><dd>All API calls made by an application can be displayed and
	include formal argument names and actual values and the
	function return value. This code is also conditionally
	included at compile time and enabled at runtime.
    </dd></dl>

    <p>The statistics and tracing can be displayed on any output
      stream (including streams opened by the shell) with output from
      different packages even going to different streams.

    </p><h2>Error Messages</h2>

    <p>By default any API function that fails will print an error
      stack to the standard error stream.

    </p><p>
      </p><center>
	<table border="" align="center" width="100%">
	  <tbody><tr>
	    <td>
	      <p><code></code></p><pre><code>
HDF5-DIAG: Error detected in thread 0.  Back trace follows.
  #000: H5F.c line 1245 in H5Fopen(): unable to open file
    major(04): File interface
    minor(10): Unable to open file
  #001: H5F.c line 846 in H5F_open(): file does not exist
    major(04): File interface
    minor(10): Unable to open file
	      </code></pre>
	    </td>
	  </tr>
	</tbody></table>
      </center>

    <p>The error handling package (H5E) is described
      <a href="./group___h5_e.html">elsewhere</a>.

    </p><h2>Invariant Conditions</h2>

    <p>To include checks for invariant conditions the library should
      be configured with <code>--disable-production</code>, the
      default for versions before 1.2. The library designers have made
      every attempt to handle error conditions gracefully but an
      invariant condition assertion may fail in certain cases.  The
      output from a failure usually looks something like this:

    </p><p>
      </p><center>
	<table border="" align="center" width="100%">
	  <tbody><tr>
	    <td>
	      <p><code></code></p><pre><code>
Assertion failed: H5.c:123: i&lt;NELMTS(H5_debug_g)
IOT Trap, core dumped.
	      </code></pre>
	    </td>
	  </tr>
	</tbody></table>
      </center>

    <h2>Timings and Statistics</h2>

    <p>Code to accumulate statistics is included at compile time by
      using the <code>--enable-debug</code> configure switch. The
      switch can be followed by an equal sign and a comma-separated
      list of package names or else a default list is used.

    </p><p>
      </p><center>
        <table border="" align="center" width="80%">
          <tbody><tr>
            <th>Name</th>
	    <th>Default</th>
            <th>Description</th>
          </tr>
	  <tr>
	    <td align="center">a</td>
	    <td align="center">No</td>
	    <td>Attributes</td>
	  </tr>
	  <tr>
	    <td align="center">ac</td>
	    <td align="center">Yes</td>
	    <td>Meta data cache</td>
	  </tr>
	  <tr>
	    <td align="center">b</td>
	    <td align="center">Yes</td>
	    <td>B-Trees</td>
	  </tr>
	  <tr>
	    <td align="center">d</td>
	    <td align="center">Yes</td>
	    <td>Datasets</td>
	  </tr>
	  <tr>
	    <td align="center">e</td>
	    <td align="center">Yes</td>
	    <td>Error handling</td>
	  </tr>
	  <tr>
	    <td align="center">f</td>
	    <td align="center">Yes</td>
	    <td>Files</td>
	  </tr>
	  <tr>
	    <td align="center">g</td>
	    <td align="center">Yes</td>
	    <td>Groups</td>
	  </tr>
	  <tr>
	    <td align="center">hg</td>
	    <td align="center">Yes</td>
	    <td>Global heap</td>
	  </tr>
	  <tr>
	    <td align="center">hl</td>
	    <td align="center">No</td>
	    <td>Local heaps</td>
	  </tr>
	  <tr>
	    <td align="center">i</td>
	    <td align="center">Yes</td>
	    <td>Interface abstraction</td>
	  </tr>
	  <tr>
	    <td align="center">mf</td>
	    <td align="center">No</td>
	    <td>File memory management</td>
	  </tr>
	  <tr>
	    <td align="center">mm</td>
	    <td align="center">Yes</td>
	    <td>Library memory management</td>
	  </tr>
	  <tr>
	    <td align="center">o</td>
	    <td align="center">No</td>
	    <td>Object headers and messages</td>
	  </tr>
	  <tr>
	    <td align="center">p</td>
	    <td align="center">Yes</td>
	    <td>Property lists</td>
	  </tr>
	  <tr>
	    <td align="center">s</td>
	    <td align="center">Yes</td>
	    <td>Data spaces</td>
	  </tr>
	  <tr>
	    <td align="center">t</td>
	    <td align="center">Yes</td>
	    <td>Datatypes</td>
	  </tr>
	  <tr>
	    <td align="center">v</td>
	    <td align="center">Yes</td>
	    <td>Vectors</td>
	  </tr>
	  <tr>
	    <td align="center">z</td>
	    <td align="center">Yes</td>
	    <td>Raw data filters</td>
	  </tr>
        </tbody></table>
      </center>

    <p>In addition to including the code at compile time the
      application must enable each package at runtime.  This is done
      by listing the package names in the <code>HDF5_DEBUG</code>
      environment variable. That variable may also contain file
      descriptor numbers (the default is `2') which control the output
      for all following packages up to the next file number.  The
      word <code>all</code> refers to all packages. Any word my be
      preceded by a minus sign to turn debugging off for the package.

    </p><p>
      </p><center>
	<table border="" align="center" width="100%">
	  <caption align="top"><b>Sample debug specifications</b></caption>
	  <tbody><tr valign="top">
	    <td><code>all</code></td>
	    <td>This causes debugging output from all packages to be
	      sent to the standard error stream.</td>
	  </tr>
	  <tr valign="top">
	    <td><code>all -t -s</code></td>
	    <td>Debugging output for all packages except datatypes
	      and data spaces will appear on the standard error
	      stream.</td>
	  </tr>
	  <tr valign="top">
	    <td><code>-all ac 255 t,s</code></td>
	    <td>This disables all debugging even if the default was to
	      debug something, then output from the meta data cache is
	      send to the standard error stream and output from data
	      types and spaces is sent to file descriptor 255 which
	      should be redirected by the shell.</td>
	  </tr>
	</tbody></table>
      </center>

    <p>The components of the <code>HDF5_DEBUG</code> value may be
      separated by any non-lowercase letter.

    </p><h2>API Tracing</h2>

    <p>The HDF5 library can trace API calls by printing the
      function name, the argument names and their values, and the
      return value. Some people like to see lots of output during
      program execution instead of using a good symbolic debugger, and
      this feature is intended for their consumption.  For example,
      the output from <code>h5ls foo</code> after turning on tracing,
      includes:

    </p><p>
      </p><center>
	<table border="" align="center" width="100%">
	  <tbody><tr>
	    <td>
	      <code><pre>
H5Tcopy(type=184549388) = 184549419 (type);
H5Tcopy(type=184549392) = 184549424 (type);
H5Tlock(type=184549424) = SUCCEED;
H5Tcopy(type=184549393) = 184549425 (type);
H5Tlock(type=184549425) = SUCCEED;
H5Fopen(filename="foo", flags=0, access=H5P_DEFAULT) = FAIL;
HDF5-DIAG: Error detected in thread 0.  Back trace follows.
  #000: H5F.c line 1245 in H5Fopen(): unable to open file
    major(04): File interface
    minor(10): Unable to open file
  #001: H5F.c line 846 in H5F_open(): file does not exist
    major(04): File interface
    minor(10): Unable to open file
	      </pre></code>
	    </td>
	  </tr>
	</tbody></table>
      </center>

    <p>The code that performs the tracing must be included in the
      library by specifying the <code>--enable-trace</code>
      configuration switch (the default for versions before 1.2). Then
      the word <code>trace</code> must appear in the value of the
      <code>HDF5_DEBUG</code> variable.  The output will appear on the
      last file descriptor before the word <code>trace</code> or two
      (standard error) by default.

    </p><p>
      </p><center>
	<table border="" align="center" width="100%">
	  <tbody><tr>
	    <td>To display the trace on the standard error stream:
	      <code><pre>$ env HDF5_DEBUG=trace a.out
	      </pre></code>
	    </td>
	  </tr>
	  <tr>
	    <td>To send the trace to a file:
	      <code><pre>$ env HDF5_DEBUG="55 trace" a.out 55&gt;trace-output
	      </pre></code>
	    </td>
	  </tr>
	</tbody></table>
      </center>

    <h3>Performance</h3>

    <p>If the library was not configured for tracing then there is no
      unnecessary overhead since all tracing code is excluded.
      However, if tracing is enabled but not used there is a small
      penalty. First, code size is larger because of extra
      statically-declared character strings used to store argument
      types and names and extra auto variable pointer in each
      function.  Also, execution is slower because each function sets
      and tests a local variable and each API function calls the
      <code>H5_trace()</code> function.

    </p><p>If tracing is enabled and turned on then the penalties from the
      previous paragraph apply plus the time required to format each
      line of tracing information.  There is also an extra call to
      H5_trace() for each API function to print the return value.

    </p><h3>Safety</h3>

    <p>The tracing mechanism is invoked for each API function before
      arguments are checked for validity.  If bad arguments are passed
      to an API function it could result in a segmentation fault.
      However, the tracing output is line-buffered so all previous
      output will appear.

    </p><h3>Completeness</h3>

    <p>There are two API functions that don't participate in
      tracing. They are <code>H5Eprint()</code> and
      <code>H5Eprint_cb()</code> because their participation would
      mess up output during automatic error reporting.

    </p><p>On the other hand, a number of API functions are called during
      library initialization and they print tracing information.

    </p><h3>Implementation</h3>

    <p>For those interested in the implementation here is a
      description.  Each API function should have a call to one of the
      <code>H5TRACE()</code> macros immediately after the
      <code>FUNC_ENTER()</code> macro.  The first argument is the
      return type encoded as a string.  The second argument is the
      types of all the function arguments encoded as a string.  The
      remaining arguments are the function arguments.  This macro was
      designed to be as terse and unobtrousive as possible.

    </p><p>In order to keep the <code>H5TRACE()</code> calls synchronized
      with the source code we've written a perl script which gets
      called automatically just before Makefile dependencies are
      calculated for the file.  However, this only works when one is
      using GNU make.  To reinstrument the tracing explicitly, invoke
      the <code>trace</code> program from the hdf5 bin directory with
      the names of the source files that need to be updated.  If any
      file needs to be modified then a backup is created by appending
      a tilde to the file name.

    </p><p>
      </p><center>
	<table border="" align="center" width="100%">
	  <caption align="top"><b>Explicit Instrumentation</b></caption>
	  <tbody><tr>
	    <td>
	      <code><pre>
$ ../bin/trace *.c
H5E.c: in function `H5Ewalk_cb':
H5E.c:336: warning: trace info was not inserted
	      </pre></code>
	    </td>
	  </tr>
	</tbody></table>
      </center>

    <p>Note: The warning message is the result of a comment of the
      form <code>/*NO TRACE*/</code> somewhere in the function
      body. Tracing information will not be updated or inserted if
      such a comment exists.

    </p><p>Error messages have the same format as a compiler so that they
      can be parsed from program development environments like
      Emacs. Any function which generates an error will not be
      modified.</p>

</body></html>
